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Description 

TECHNICAL FIELD 

5 [0001] The invention relates to an apparatus, algorithm, and method for transforming a hemispherical field-of-view 
image into a non-distorted, normal perspective image at any orientation, rotation, and magnification within the field-of- 
view. The viewing direction, orientation, and magnification are controlled by either computer or remote control means. 
More particularly, this apparatus is the electronic equivalent of a mechanical pan, tilt, zoom, and rotation camera viewing 
system with no moving mechanisms. 

10 

BACKGROUND ART 

[0002] Camera viewing systems are utilized in abundance for surveillance, inspection, security, and remote sensing. 
Remote viewing is critical for robotic manipulation tasks. Close viewing is necessary for detailed manipulation tasks 

is while wide-angle viewing aids positioning of the robotic system to avoid collisions with the work space. The majority 
of these systems use either a fixed-mount camera with a limited viewing field, or they utilize mechanical pan-and-tilt 
platforms and mechanized zoom lenses to orient the camera and magnify its image. In the applications where orien- 
tation of the camera and magnification of its image are required, the mechanical solution is large and can subtend a 
significant volume making the viewing system difficult to conceal or use in close quarters. Several cameras are usually 

20 necessary to provide wide-angle viewing of the work space. 

[0003] In order to provide a maximum amount of viewing coverage or subtended angle, mechanical panAitt mecha- 
nisms usually use motorized drives and gear mechanisms to manipulate the vertical and horizontal orientation. An 
example of such a device is shown in U.S. Patent Number 4,728,839 issued to J.B. Coughlan, et al, on March 1 , 1988. 
Collisions with the working environment caused by these mechanical pan/tilt orientation mechanisms can damage both 

25 the camera and the work space and impede the remote handling operation. Simultaneously, viewing in said remote 
environments is extremely important to the performance of inspection and manipulation activities. 
[0004] Camera viewing systems that use internal optics to provide wide viewing angles shave also been developed 
in order to minimize the size and volume of the camera and the intrusion into the viewing area. These systems rely on 
the movement of either a mirror or prism to change the tilt-angle of orientation and provide mechanical rotation of the 

30 entire camera to change the pitch angle of orientation. Using this means, the size of the camera orientation system 
can be minimized, but "blind spats" In the center of the view result. Also, these systems typically have no means of 
magnifying the image and or producing multiple images from a single camera. 

[0005] JP-02-1 27877A of Casio describes a system for correcting the distortion in any of nine predefined regions of 
an image captured by a fish eye lens. Distortion data used for the correction is pre-stored in memory. 
35 [0006] EP-A-01 1 1 909 describes another picture processing apparatus for correcting a distorted picture captured by 
a wide angle lens. 

[0007] WO 820371 2A describes a system for transforming an image. Input commands designating X and Y pre- 
translation, X and Y size control, 2 axis rotation and X and Y post-translation are received and used to generate a 
composite transformation factored into horizontal and vertical components which are applied to the image. 
40 [0008] In the paper 'Movie Maps: An Application of the Optical Video Disc to Computer Graphics' (Siggraph confer- 
ence proceedings, 1 980, pages 32 to 42), LIPPMAN A. describes a videodisc system which manipulates stored images 
to enable a user to experience a simulated 'drive' through a space. 

[0009] Accordingly, it is an abject of the present invention to provide an apparatus that can provide an image of any 
portion of the viewing space within a hemispherical field-of-view without moving the apparatus. 
45 [001 0] It is another object of the present invention to provide horizontal orientation (pan) of the viewing direction with 
no moving mechanisms. 

[001 1] It is another object of the present invention to provide vertical orientation (tilt) of the viewing direction with no 
moving mechanisms. 

[0012] It is another object of the present invention to provide rotational orientation (rotation) of the viewing direction 
so with no moving mechanisms. 

[001 3] It is another object of the present invention to provide the ability to magnify or scale the image (zoom in and 
out) electronically. 

[0014] It is another object of the present invention to provide electronic control of the image intensity (iris level) 
[0015] It is another abject of the present invention to be able to change the image intensity (iris level) without any 
55 mechanisms. 

[0016] It is another object of the present invention to be able to accomplish said pan, tilt, zoom, rotation, and iris with 

simple inputs made by a lay person from a joystick, keyboard controller, or computer controlled means. 

[0017] It is also an object of the present invention to provide accurate control of the absolute viewing direction and 
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orientations using said input devices. 

[0018] A further object of the present invention is to provide the ability to produce multiple images with different 
orientations and magnifications simultaneously. 

[0019] Another object of the present invention is to be able to provide these images at real-time video rates, that is 
s 30 transformed images per second, and to support various display format standards such as the National Television 
Standards Committee RS-170 display format, 

[0020] These and other objects of the present invention will become apparent upon consideration of the drawings 
hereinafter in combination with a complete description thereof. 

10 DISCLOSURE OF THE INVENTION 

[0021] The present invention provides a system as set forth in claim 1 and a method as set forth in claim 13. 
[0022] In one preferred embodiment, the incoming image is produced by a fisheye lens which has a hemispherical 
field-of-view. This hemispherical field-of-view image is captured into an electronic memory buffer. A portion of the 

15 captured image containing a region-of-interest is transformed into a perspective correct image by image processing 
computer means. The image processing computer provides direct mapping of the hemispherical image region-of- 
interest into a corrected image using an orthogonal set of transformation algorithms. The viewing orientation is desig- 
nated by a command signal generated by either a human operator or computerized input. The transformed image is 
deposited in a second electronic memory buffer where it is then manipulated to produce the output image as requested 

20 by the command signal. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0023] 

25 

FIG. 1 shows a schematic block diagram of the present invention illustrating the major components thereof. 
FIG. 2 is an example sketch of a typical fisheye image used as input by the present invention. 
FIG. 3 is an example sketch of the output image after correction for a desired image orientation and magnification 
within the original image. 

30 FIG. 4 is a schematic diagram of the fundamental geometry that the present invention embodies to accomplish 

the image transformation. 

FIG. 5 is a schematic diagram demonstrating the projection of the object plane and position vector into image 
plane coordinates. 

35 BEST MODE FOR CARRYING OUT THE INVENTION 

[0024] In order to minimize the size of the camera orientation system while maintaining the ability to zoom, a camera 
orientation system that utilizes electronic image transformations rather than mechanisms was developed. While nu- 
merous patents on mechanical pan-and-tilt systems have been filed, no approach using strictly electronic transforms 
40 and fisheye optics has ever been successfully implemented prior to this effort. In addition, the electro-optical approach 
utilized in the present invention allows multiple images to be extracted from the output of a single camera. Motivation 
for this device came from viewing system requirements in remote handling applications where the operating envelop 
of the equipment is a significant constraint to task accomplishment. 

[0025] The principles of the present invention can be understood by reference to FIG. 1 . Shown schematically at 1 
45 is the fisheye lens that provides an image of the environment with a 180 degree field-of-view. The fisheye lens is 
attached to a camera 2 which converts the optical image into an electrical signal. These signals are then digitized 
electronically 3 and stored in an image buffer 4 within the present invention. An image processing system consisting 
of an X-MAP and a Y-MAP processor shown as 6 and 7, respectively, performs the two-dimensional transform mapping. 
The image transform processors are controlled by the microcomputer and control interface 5. The microcomputer 
so control Interface provides initialization and transform parameter calculation for the system. The control interface also 
determines the desired transformation coefficients based on orientation angle, magnification, rotation, and light sen- 
sitivity input from an input means such as a joystick controller 1 2 or computer input means 1 3. The transformed image 
is filtered by a 2-dimensional convolution filter 8 and the output of the filtered image is stored in an output image buffer 
9. The output image buffer 9 is scanned out by display electronics 1 0 to a video display device 11 for viewing. 
55 [0026] A range of lens types can be accommodated to support various fields of view. The lens optics 1 correspond 
directly with the mathematical coefficients used with the X-MAP and Y-MAP processors 6 and 7 to transform the image. 
The capability to pan and tilt the output image remains even though a different maximum field of view is provided with 
a different lens element. 
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[0027] The invention can be realized by proper combination of a number of optical and electronic devices. Thefisheye 
lens 1 is exemplified by any of a series of wide angle lenses from, for example, Nikon, particularly the 8mm F2.8. Any 
video source 2 and image capturing device 3 that converts the optical image into electronic memory can serve, as the 
input for the invention such as a Videk Digital Camera interfaced with Texas Instrument's TMS 34061 integrated circuits. 

5 Input and output image buffers 4 and 9 can be constructed using Texas Instrument TMS44C251 video random access 
memory chips or their equivalents. The control interface can be accomplished with any of a number of microcontrollers 
including the Intel 80C1 96. The X-MAP and Y-MAP transform processors 6 and 7 and image filtering 8 can be accom- 
plished with application specific integrated circuits or other means as will be known to persons skilled in the art. The 
display driver can also be accomplished with integrated circuits such as the Texas Instruments TMS34061 . The output 

10 video signal can be of the NTSC RS-1 70, for example, compatible with most commercial television displays in the 
United States. Remote control 12 and computer control 13 are accomplished via readily available switches and/or 
computer systems that also will be well known. These components function as a system to select a portion of the input 
image (fisheye or wide angle) and then mathematically transform the image to provide the proper prospective for output. 
The keys to the success of the invention include: 

15 

(1) the entire input image need not be transformed, only the portion of interest 

(2) the required mathematical transform is predictable based on the lens characteristics. 

[0028] The transformation that occurs between the input memory buffer 4 and the output memory buffer 9, as con- 

20 trolled by the two coordinated transformation circuits 6 and 7, is better understood by looking at FIG. 2 and FIG. 3, The 
image shown in FIG. 2 is a pen and ink rendering of the image of a grid pattern produced by a fisheye lens, This image 
has a field-of-view of 1 80 degrees and shows the contents of the environment throughout an entire hemisphere. Notice 
that the resulting image in FIG. 2 is significantly distorted relative to human perception. Vertical grid lines in the envi- 
ronment appear in the image plane as 14a, 14b, and 14c. Horizontal grid lines in the the envimoment appear in the 

25 image plane as 15a, 15b, and 15c. The image of an object is exemplified by 16. A portion of the image in FIG. 2 has 
been correct, magnified, and rotated to produce the image shown in FIG. 3. Item 17 shows the corrected representation 
of the object in the output display. The results shown in the image in FIG. 3 can be produced from any portion of the 
image of FIG. 2 using the present invention. Note the corrected perspective as demonstrated by the straightening of 
the grid pattern displayed in FIG. 3. In the present invention, these transformations can be performed at real-time video 

30 rates (30 times per second), compatible with commercial video standards. 

[0029] The invention as described has the capability to pan and tilt the output image through the entire field of view 
of the lens element by changing the input means, e.g. the joystick or computer, to the controller. This allows a large 
area to be scanned for information as can be useful in security and surveillance applications. The image can also be 
rotated through 360 degrees on its axis changing the perceived vertical of the displayed image. This capability provides 

35 the ability to align the vertical image with the gravity vector to maintain a proper perspective in the image display 
regardless of the pan or tilt angle of the image. The invention also supports modifications in the magnification used to 
display the output image. This is commensurate with a zoom function that allows a change in the field of view of the 
output image. This function is extremely useful for inspection operations. The magnitude of zoom provided is a function 
of the resolution of the input camera, the resolution of the output display, the clarity of the output display, and the amount 

40 of picture element (pixel) averaging that is used in a given display. The invention supports all of these functions to 
provide capabilities associated with traditional mechanical pan (through 180 degrees), tilt (through 180 degrees), ro- 
tation (through 360 degrees), and zoom devices. The digital system also supports image intensity scaling that emulates 
the functionality of a mechanical iris by shifting the intensity of the displayed image based on commands from the user 
or an external computer. 

45 [0030] The postulates and equations that follow are based on the present invention utilizing a fisheye lens as the 
optical element. There are two basic properties and two basic postulates that describe the perfect fisheye lens system. 
The first property of a fisheye lens is that the lens has a 2n steradian field-of-view and the image it produces is a circle. 
The second property is that all objects in the field-of-view are in focus, i.e. the perfect fisheye lens has an infinite depth- 
of-field. The two important postulates of the fisheye lens system (refer to Figures 4 and 5) are stated as follows: 

50 

Postulate 1 : Azimuth angle invariability - For object points that lie in a content plane that is perpendicular to the 
image plane and passes through the image plane origin, all such points are napped as image points onto the line 
of intersection between the image plane and the content plane, i.e. along a radial line. The azimuth angle of the 
image points is therefore invariant to elevation and object distance changes within the content plane. 
55 Postulate 2: Equidistant Projection Rule - The radial distance, r, from the image plane origin along the azimuth 

angle containing the projection of the object point is linearly proportional to the zenith angle p, where p is defined 
as the angle between a perpendicular line through the image plane origin and the line from the image plane origin 
to the object point. Thus the relationship: 
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r-kp (1) 

[0031] Using these properties and postulates as the foundation of the fisheye lens system, the mathematical trans- 
5 formation for obtaining a perspective corrected image can be determined. FIG. 4 shows the coordinate reference 
frames for the object plane and the image plane. The coordinates u f v describe object points within the object plane. 
The coordinates x,y,z describe points within the image coordinate frame of reference. 

[0032] The object plane shown in FIG. 4 is a typical region of interest to determine the napping relationship onto the 
image plane to properly correct the object. The direction of view vector, DOV[x,y,z], determines the zenith and azimuth 
10 angles for mapping the object plane, UV, onto the image plane, XY The object plane is defined to be perpendicular to 
the vector, DOV[x,y,z]. 

[0033] The location of the origin of the object plane in terms of the image plane [x,y,z] in spherical coordinates is 
given by: 

15 

x = D sinp cos3 
y = Dsinpsin3 

z = D cosp (2) 

where D = scalar length from the image plane origin to the object plane origin, (3 is the zenith angle, and 3 is the azimuth 
angle in image plane spherical coordinates. The origin of object plane is represented as a vector using the components 
given in equation 1 as: 

DOV[x,y,z] = [Dsinpcosa, Dsinpsin3, Dcosp] (3) 

[0034] DOV[x,y,z] is perpendicular to the object plane and its scalar magnitude D provides the distance to the object 
30 plane. By aligning the YZ plane with the direction of action of DOV[x,y,z], the azimuth angle d becomes either 90 or 
270 degrees and therefore the x component becomes zero resulting in the DOV[x,y,z] coordinates: 
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55 



DOV[x,y,z] = [0, -Dsinp, Dcosp] (4) 

[0035] Referring now to FIG. 5, the object point relative to the UV plane origin in coordinates relative to the origin of 
the image plane is given by the following: 

x = u 
y = vcosp 

z = vsinp (5) 

therefore, the coordinates of a point P(u,v) that lies in the object plane can be represented as a vector P[x,y,z] in image 
plane coordinates: 

P[x,y,z] = [u, vcosp, vsinp] (6) 

where P[x,y,z] describes the position of the object point in image coordinates relative to the origin of the UV plane. The 
object vector o[x,y,z] that describes the object point in image coordinates is then given by: 

0[x,y,z] = DOV[x,y,z] + P[x,y,z] (7) 
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0[x,y,z] = [u, vcosp - Dsinp, vsinp + Dcosp] (8) 

[0036] Projection onto a hemisphere of radius R attached to the image plane is determined by scaling the object 
5 vector o[x,y,z] to produce a surface vector s[x,y,z]: 

Sfrvzl- R0[x ' y ' z ) <9) 
[ ,y " 1 " |0[x ( y,z]| (9) 

10 [0037] By substituting for the components of 0[x,y,z] from Equation 8, the vector S[x,y,z] describing the image point 
mapping onto the hemisphere becomes: 

S[x j = RO[u, (vcosp - Dsinp), (vsinp + Dcosp)] {1 Q) 

Vu 2 +(vcosp - Dsinp) 2 , (vsinp + Dcosp) 2 



15 



20 



25 



30 



[0038] The denominator in Equation 10 represents the length or absolute value of the vector 0[x,y,z] and can be 
simplified through algebraic and trigonometric manipulation to give: 

S[x,y,z] - RQ[U ' (vcosp ' P 8 '"^' (vsin P + Pcos P)] (11) 
7u 2 +v 2 +D 2 

[0039] From equation 1 1 , the mapping onto the two-dimensional image plane can be obtained for both x and y as: 

Ru 



•fx 



u 2 + v 2 + D 2 



(12) 



R(vcosp - DsinP) (13) 

[0040] Additionally, the image plane center to object plane distance D can be represented in terms of the fisheye 
35 image circular radius R by the relation: 

D=mR (14) 

40 where m represents the scale factor in radial units R from the image plane origin to the object plane origin. 

Substituting Equation 14 into Equations 12 and 13 provides a means for obtaining an effective scaling operation or 
magnification which can be used to provide zoom operation. 
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" , Ru (15) 



Vu 2 - 2 - 2 " 2 



+v +m R 



y ^ R(vcosp-mRsinp) (16) 
Vu 2 +v 2 +m 2 R 2 

[0041] Using the equations for two-dimensional rotation of axes for both the UV object plane and the XY image plane 
the last two equations can be further manipulated to provide a more general set of equations that provides for rotation 
within the image plane and rotation within the object plane. 

x _ R[uA-vB+mRsinpsin3] ^-^ 
Ju 2 +v 2 +m 2 R 2 



6 



EP 0 971 540 B1 



„ R[uC-vD+mRsinf5sin8] ,, QX 
y= ====== [\o) 



7^2 . .2 _2„2 



+v +m R 

where: 



A= (cos0cos8 - sin0sindcosp) 
1Q B = (sin0cos3 + cos0sin3cosP) 

C = (cos0sin3 + sin0cos3cosP) 

D = (sin0sin3 - cos0cos9cosf}) (19) 

15 

and where: 

R = radius of the image circle 
P = zenith angle 
20 3 = Azimuth angle in image plane 

0 = Object plane rotation angle 
m = Magnification 
u,v = object plane coordinates 
x,y = image plane coordinates 

25 

[0042] The Equations 17 and 18 provide a direct mapping from the UV space to the XY image space and are the 
fundamental mathematical result that supports the functioning of the present omnidirectional viewing system with no 
moving parts. By knowing the desired zenith, azimuth, and object plane rotation angles and the magnification, the 
locations of x and y in the imaging array can be determined. This approach provides a means to transform an image 
30 from the input video buffer to the output video buffer exactly. Also, the fisheye image system is completely symmetrical 
about the zenith, therefore, the vector assignments and resulting signs of various components can be chosen differently 
depending on the desired orientation of the object plane with respect to the image plane. In addition, these postulates 
and mathematical equations can be modified for various lens elements as necessary for the desired fieid-of-view cov- 
erage in a given application. 

35 [0043] The input means defines the zenith angle, p, the azimuth angle, 3, the object rotation, 0, and the magnification , 
m. These values are substituted into Equations 19 to determine values for substitution into Equations 17 and 18. The 
image circle radius, R, is a fixed value that is determined by the camera lens ane element relationship. The variables 
u and v vary throughout the object plane determining the values for x and y in the image plane coordinates. 
[0044] From the foregoing, it can be seen that a fisheye lens provides a hemispherical view that is captured by a 

40 camera. The image is then transformed into a corrected image at a desired pan, tilt, magnification, rotation, and focus 
based on the desired view as described by a control input. The image is then output to a television display with the 
perspective corrected. Accordingly, no mechanical devices are required to attain this extensive analysis and presen- 
tation of the view of an environment through 180 degrees of pan, 180 degrees of tilt, 360 degrees of rotation, and 
various degrees of zoom magnification. 

45 

Claims 

1 . A system for providing perspective corrected views of a selected portion of a received optical image captured using 
so a wide angle lens (1 ), the received optical image being distorted, the system comprising: 

image capture means (3) for receiving signals corresponding to said received optical image and for digitising 
said signal; 

input image memory means (4) for receiving said digitised signal; 
55 input means (12,13) for selecting a portion of said received image to view; 

image transform processor means (5) for processing said digitised signals to produce an output signal corre- 
sponding to a perspective corrected image of said selected portion of said received image; 
output image memory means (9) for receiving said output signal from said image transform processor means 
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(5); and 

output means (10,11) connected to said output image memory means (9) for recording or displaying said 
perspective corrected image of said selected portion; 

CHARACTERISED IN THAT 

said image transform processor means (5) comprises transform parameter calculation means for calculating 
transform parameters for said selected portion of said image and processes said digitised signal based on said 
calculated transform parameters to generate said output signal. 

2. A system according to claim 1 , comprising a camera imaging system (2) for receiving said optical image and for 
producing said signals corresponding to said received optical image for output to said image capture means (3). 

3. A system according to claim 2, comprising wide angle lens means (1) mounted on said camera imaging system 
for producing said optical image for optical conveyance to said camera imaging system. 

4. A system according to claim 3, wherein said lens means is one or more fish-eye lenses (1). 

5. A system according to any one of the preceding claims, wherein said input means (12,13) provides for input to 
said image transform processor means of one or more of: a direction of view; tilting of a viewing angle; rotation of 
a viewing angle; pan of said viewing angle; focus of said image and magnification of the selected portion of the 
image. 

6. A system according to claim 5, wherein tilting of said viewing angle through at least 180 degrees is provided for. 

7. A system according to claim 5 or claim 6, wherein rotation of said viewing angle through 360 degrees is provided for. 

8. A system according to any one of claims 5 to 7, wherein pan of said viewing angle through at least 1 80 degrees 
is provided for. 

9. A system according to claim 8, wherein pan of said viewing angle through 360 degrees is provided for. 

10. A system according to any one of the preceding claims, wherein said input means is a user-operated manipulator 
switch means (12). 

1 1 . A system according to any one of claims 1 to 9, wherein said input means is a signal from a computer input means 



12. A system according to any one of the preceding claims, wherein said image transform processing means (5) is 
programmed to implement the following two equations: 



(13). 



R{uA-vB+mRsinpsin8} 



Vu 2 +v 2 +m 2 R 2 



R{uC-vD+mRsinpsin9} 



7u 2 +v 2 +m 2 R 2 



where: 



A 



(cos (f) cos 6 - sin<t>sinScosp) 



B 



(sin<|>cosS + cos(|>sinScosfS) 



C 



(cos(J)Sin5 + sin<t>cos5cos[3) 
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D = (sin<t>sin5 + cos(|>cos8cosp) 

and where: 

5 

R = radius of the image circle 
p = zenith angle 

S = Azimuth angle in image plane 
<|> = Object plane rotation angle 
10 m = Magnification 

u,v = object plane coordinates 
x,y = image plane coordinates 

13. A method for providing perspective corrected views of a selected portion of an optical image captured with a wide 
15 angle lens (1), the received optical image being distorted, the method comprising: 

providing a digitised signal corresponding to said optical image; 
selecting a portion of said optical image; 

transforming said digitised signal to produce an output signal corresponding to a perspective corrected image 
20 of said selected portion of said received image; and 

displaying or recording said perspective corrected image of said selected portion; 

CHARACTERISED IN THAT 

said step of transforming said digitised signal comprises calculating transform parameters for said selected 
25 portion of said image, said calculated transform parameters being used to control said transformation of the digi- 

tised signal to generate said output signal. 

14. A method according to claim 13, comprising first receiving said optical image, producing signals corresponding to 
said received optical image and digitizing said signals. 



30 



55 



1 5. A method according to claim 1 3 or 1 4, comprising capturing said optical image with one or more fish-eye lenses (1 ). 



1 6. A method according to any one of claims 1 3 to 1 5, wherein said step of selecting the portion of the image to view 
comprises selecting one or more of: a direction of view; tilting of a viewing angle; rotation of a viewing angle; pan 

35 of said viewing angle; focus of said image and magnification of the selected portion of the image. 

17. A method according to claim 16, wherein tilting of said viewing angle through at least 1 80 degrees is provided for. 

1 8. A method according to claim 1 6 or claim 1 7, wherein rotation of said viewing angle through 360 degrees is provided 
40 for. 

1 9. A method according to any one of claims 1 6 to 1 8, wherein pan of said viewing angle through at least 1 80 degrees 
is provided for. 

45 20. A method according to claim 1 9, wherein pan of said viewing angle through 360 degrees is provided for. 

21. A method according to any one of claims 13 to 20, wherein said selection of said portion of the image to view is 
achieved using a user-operated manipulator switch means (12). 

so 22. A method according to any one of claims 1 3 to 20, wherein said selection of said portion of the image to view is 
controlled by a signal from a computer input means (13). 

23. A method according to any one of claims 13 to 22, wherein said image transformation implements the following 
two equations: 



x _ R{uA-vB-HT)Rsin[3sin8} 



!~2 2 2 0 2 

Vu +v +m R 
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where: 



A = (costJicosS - sin<|>sin8cosP) 



B 



(si n^) cos 5 + cos<j>sin8cosP) 



C 



(cos<t>sinS + sin(f)cos8cosp) 



D 



(sin<t»sin5 + cos<|>cos8cosP) 



and where: 

R = radius of the image circle 
p = zenith angle 

8 = Azimuth angle in image plane 
<|> = Object plane rotation angle 
m = Magnification 
u,v = object plane coordinates 
x,y = image plane coordinates 

24. A method according to any one of claims 13 to 23, wherein a plurality of portions of said image are selected for 
viewing and are displayed either simultaneously or consecutively. 

25. A method according to any one of claims 1 3 to 24, wherein the image is viewed interactively by repeating the steps 
of selecting, transforming and displaying said portion of the image. 

26. A method according to claim 13, wherein said step of transforming the image is based on lens characteristics of 
the wide angle lens (1). 

27. A method according to claim 26, wherein the step of transformation is based on azimuth angle invariability and 
equidistant projection. 

28. A method according to claim 1 3, wherein the step of transforming the image is performed at real time video rates. 



Patentanspruche 

1 . System zum Vorsehen perspektivisch korrigierter Ansichten eines ausgewahlten Teils eines empfangenen opti- 
schen Bildes, das mit einer Weitwinkellinse (1) aufgenommen ist, wobei das empfangene optische Bild verzerrt 
ist, mitfolgenden Merkmalen: 

eine Bilderfassungseinrichtung (3) zum Empfangen von Signalen, welche dem empfangenen optischen Bild 
entsprechen, und zum Digitalisieren des Signals; 

eine Eingangsbildspeichereinrichtung (4) zum Empfangen des digitalisierten Signals; 

eine Eingabeeinrichtung (12, 13) zum Auswahlen eines Teils des empfangenen Bildes fur die Betrachtung; 

eine Bildtransformationsverarbeitungseinrichtung (5) zum Verarbeiten der digitalisierten Signale zum Erzeu- 
gen eines Ausgangssignals, welches einem perspektivisch korrigierten Bild des ausgewahlten Teils des emp- 
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fangenen Bildes entspricht; 

eine Ausgangsbildspeichereinrichtung (9) zum Empfangen des Ausgangssignals von der Bildtransformations- 
verarbeitungseinrichtung (5); und 

eine Ausgabeeinrichtung (10, 11), die mit der Ausgangsbildspeichereinrichtung (9) verbunden ist, zum Auf- 
zeichnen Oder Anzeigen des perspektivisch korrigierten Bildes des ausgewahlten Teils; 

dadurch gekennzeichnet,daB 

die Bildtransformattonsverarbeitungseinrichtung (5) eine Transformationsparameterrecheneinrichtung zum Be- 
rechnen von Transformationsparametern fur den ausgewahlten Teil des Bildes umfaBt und das digitalisierte Signal 
gestiitzt auf die berechneten Transformationsparameter verarbeitet, urn das Ausgangssignal zu erzeugen. 

2. System nach Anspruch 1 , mit einem Kam e raabbi Id ungs system (2) zum Empfangen des optischen Bildes und zum 
Erzeugen der Signale, welche dem empfangenen optischen Bild entsprechen, zur Ausgabe an die Bilderfassungs- 
einrichtung (3). 

3. System nach Anspruch 2, mit einer Weitwinkellinseneinrichtung (1) t die an dem Kameraabbildungssystem ange- 
bracht ist, zum Erzeugen des optischen Bildes fur die optische Ubertragung an das Kameraabbildungssystem. 

4. System nach Anspruch 3, wobei die Linseneinrichtung eine oder mehrere Fischaugen-Linsen (1) umfaBt. 

5. System nach einem der vorangehenden Anspriiche, wobei die Eingabeeinrichtung (12, 13) eine Eingabe fur die 
Bildtransformationsverarbeitungseinrichtungaus einem odermehreren derfolgenden vorsieht: Blickrichtung; Nei- 
gung eines Blickwinkels; Drehung eines Blickwinkels; Schwenken des Blickwinkels; Brennpunkt des Bildes und 
VergroBerung des ausgewahlten Teiles des Bildes. 

6. System nach Anspruch 5, wobei das Neigen des Blickwinkels urn wenigstens 180° vorgesehen ist. 

7. System nach Anspruch 5 oder Anspruch 6, wobei die Drehung des Blickwinkels urn 360° vorgesehen ist. 

8. System nach einem der Anspriiche 5 bis 7, wobei das Schwenken des Blickwinkels um wenigstens 180° vorge- 
sehen ist. 

9. System nach Anspruch 8, wobei das Schwenken des Blickwinkels um 360° vorgesehen ist. 

10. System nach einem der vorangehenden Anspriiche, wobei die Eingabeeinrichtung eine von einem Benutzer be- 
dienbare Einstell-Schalteinrichtung (12) ist. 

11. System nach einem der Anspriiche 1 bis 9, wobei die Eingabeeinrichtung ein Signal von einer Computereingabe- 
einrichtung (13) vorsieht. 

12. System nach einem der vorangehenden Anspriiche, wobei die Bildtransfomnationsverarbeitungseinrichtung (5) 
programmiert ist, um die folgenden zwei Gleichungen zu realisieren 

_ R[uA-vB+mRs\r\$s\nB] 

y= fl["C-vP+roflsinpsin3] 
Ju 2 +v 2 +m 2 F? 

wobei 

A = (cos0cos3 - sin0sindcosp) 
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B = (sin0cos3 + cos0 sin3cosp) 



C = (cos0sin3 + sin0cos8cosp) 



D = (sin0sin9 + cos0cos9cosp) 

und wobei 

R = Radius des Bildkreises 
P = Zenitwinkel 

B = Azimuthwinkel in Bildebene 
0 - Drehwinkel der Objektebene 
m = VergroBerung 
u, v = Koordinaten in Objektebene 
x, y = Koordinaten in Bildebene. 

13. Verfahren zum Vorsehen perspektivisch korrigierter Ansichten eines ausgewahiten Teils eines optischen Bildes, 
das mit einer Weitwinkellinse (1) erfaBt wird, wobei das empfangene optische Bild verzerrt ist, mitfolgenden Ver- 
fahrensschritten: 

Vorsehen eines digitalisierten Signals, das dem optischen Bild entspricht; 
Auswahlen eines Teils des optischen Bildes; 

Transformteren des digitalisierten Signals zum Erzeugen eines Ausgangssignals, das einem perspektivisch 
korrigierten Bild des ausgewahiten Teils des empfangenen Bildes entspricht; und 

Anzeigen oder Aufzeichnen des perspektivisch korrigierten Bildes des ausgewahiten Teils; 

dadurch gekennzeichnet, daB 

der Schritt des Transform ierens des digitalen Signals das Berechnen von Transformationsparametern fur den 
ausgewahiten Teil des Bildes umfaBt, wobei die berechneten Transformationsparameter dazu verwendet werden, 
die Transformation des digitalisierten Signals zum Erzeugen des Ausgangssignals zu steuern. 

14. Verfahren nach Anspruch 13, bei dem zuerst das optische Bild empfangen und Signale erzeugt werden, wetche 
dem empfangenen optischen Bild entsprechen, und diese Signale digitalisiert werden. 

15. Verfahren nach Anspruch 13 oder 14, bei dem das optische Bild mit einer oder mehreren Fischaugen-Linsen (1) 
erfaBt wird. 

1 6. Verfahren nach einem der Anspruche 1 3 bis 1 5, wobei der Schritt des Auswahlens des Teils bis zur Betrachtung 
des Bildes das Auswahlen eines oder mehrerer der folgenden umfaBt: Blickrichtung; Neigung eines Blickwinkels; 
Drehung eines Blickwinkels; Schwenken des Blickwinkels; Brennpunkt des Bildes und VergroBerung des ausge- 
wahiten Teiles des Bildes. 

17. Verfahren nach Anspruch 16, wobei der Blickwinkel urn wenigstens 180° geneigt werden kann. 

18. Verfahren nach Anspruch 16 oder Anspruch 17, wobei der Blickwinkel urn 360° gedreht werden kann. 

19. Verfahren nach einem der Anspriiche 16 bis 18, wobei der Blickwinkel urn wenigstens 180° geschwenkt werden 
kann. 

20. Verfahren nach Anspruch 19, wobei der Blickwinkel urn 360° geschwenkt werden kann. 

21. Verfahren nach einem der Anspruche 13 bis 20, wobei das Auswahlen des Teils des zu betrachtenden Bildes 
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mittels einer von einem Benutzer bedienten Einstell-Schalteinrichtung (12) durchgefuhrt wird, 

22. Verfahren nach einem der Anspriiche 13 bis 20, wobei das Auswahlen des Teils des zu betrachtenden Bildes 
durch ein Signal von einer Computereingabeeinrichtung (13) gesteuert wird. 

23. Verfahren nach einem der Anspriiche 13 bis 22, wobei die Biotransformation die folgenden zwei Gleichungen 
realisiert: 



10 



R[uA-vB+mRs\r)$$\r\d] 
Ju 2 +^+m 2 Pp 



15 



f _ R[uC-vD+mR$'m$s\r)d] 



wobei 



20 



A = (cos0cos3 - sin0sin3cosp) 



B = (sin0cosd + cos0 sindcosP) 



25 



C = (cos0sin8 + sin0cosdcos[}) 



30 



und wobei 



D = (sin0sin8 + cos0cosdcosfJ) 



R = Radius des Bildkreises 
p = Zenitwinkel 

d = Azimuthwinkel in Bildebene 
0 = Drehwinkel der Objektebene 
m = VergroBerung 
u, v = Koordinaten in Objektebene 
x, y = Koordinaten in Bildebene. 

24. Verfahren nach einem der Anspriiche 13 bis 23, wobei mehrere Teile des Bildes fur die Betrachtung ausgewahlt 
und entweder gleichzeitig oder nacheinander angezeigt werden. 

25. Verfahren nach einem der Anspriiche 13 bis 24, wobei das Bild interaktiv betrachtet wird, indem die Schritte des 
Auswahlens, Transformierens und Anzeigen des Teiles des Bildes wiederholt werden. 

26. Verfahren nach Anspruch 13, wobei der Schritt des Transformierens des Bildes auf Linseneigenschaften der Weit- 
winkellinse (1) basiert. 

so 27. Verfahren nach Anspruch 26, wobei der Schritt des Transformierens auf einer Invariabiiitat des Azimuthwinkels 
und einer aquidistanten Projektion basiert. 

28. Verfahren nach Anspruch 13, wobei der Schritt des Transformierens des Bildes mit Realzeit-Videoraten durchge- 
fuhrt wird. 

55 



35 



40 



45 
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Revendications 

1 . Systeme permettant de fournir des vues corrigees en perspective d'une partie choisie d'une image optique recue 
capturee a I'aide d'un objectif grand angulaire (1), I'image optique recue etant deformee, le systeme comprenant : 

5 

des moyens de capture d'image (3) permettant de recevoir des signaux correspondant a ladite image optique 
recue et permettant de numeriser ledit signal ; 

des moyens de memoire d'image d'entree (4) permettant de recevoir le signal numerise ; 
des moyens d'entree (12, 13) permettant de choisir une partie de ladite image recue a visualiser ; 
10 des moyens de processeur de transformation d'image (5) permettant de traiter lesdits signaux numerises afin 

de produire un signal de sortie correspondant a une image corrigee en perspective de ladite partie choisie de 
ladite image recue ; 

des moyens de memoire d'image de sortie (9) permettant de recevoir ledit signal de sortie desdits moyens de 
processeur de transformation d'image (5) ; et 
15 des moyens de sortie (10,11) relies aux dits moyens de memoire d'image de sortie (9) permettant d'enregistrer 

ou de montrer ladite image corrigee en perspective de ladite partie choisie ; 

CARACTERISE EN CE QUE 

lesdits moyens de processeur de transformation d'image (5) comprennent des moyens de calcul de para- 
20 metres de transformation permettant de calcuter les parametres de transformation de ladite partie choisie de ladite 

image, et traitent ledit signal numeris6 sur la base desdits parametres de transformation calcules afin de produire 
ledit signal de sortie. 

2. Systeme selon la revendication 1, comprenant un systeme de formation d'image par camera (2) permettant de 
25 recevoir ladite image optique et de produire lesdits signaux correspondant a ladite image optique recue, a desti- 
nation desdits moyens de capture d'image (3). 

3. Systeme selon la revendication 2, comprenant des moyens d'objectif grand angulaire (1 ) montes sur ledit systeme 
de formation d'image par camera afin de produire ladite image optique pour un transfert optique vers ledit systeme 

30 de formation d'image par camera. 

'4. Systeme selon la revendication 3, dans lequel lesdits moyens d'objectif sont constitues de un ou de plusieurs 
objectifs ultra grand angulaires (1). 

35 5. Systeme selon Tune quelconque des revendications precedentes, dans lequel lesdits moyens d'entree (12, 13) 
fournissent a Pentree desdits moyens de processeur de transformation d'image, Tun des elements suivants, ou 
plusieurs d'entre eux : une direction de vis6e ; I'inclinaison d'un angle de visee ; la rotation d'un angle de visee ; 
le panoramique dudit angle de visee ; la mise au point de ladite image et le grossissement de la partie choisie de 
I'image. 

40 

6. Systeme selon la revendication 5, dans lequel I'inclinaison dudit angle de visee est prevue sur au moins 180 °. 

7. Systeme selon Tune quelconque des revendications 5 ou 6, dans lequel la rotation dudit angle de visee est prevue 
sur 360 °. 

45 

8. Systeme selon Tune quelconque des revendications 5 a 7, dans lequel le panoramique dudit angle de visee est 
prevu surau moins 180 °. 

9. Systeme selon la revendication 8, dans lequet le panoramique dudit angle de visee est prevu sur 360 °. 

50 

10. Systeme selon Tune quelconque des revendications precedentes, dans lequel lesdits moyens d'entree sont des 
moyens de commutateur de manipulateur actionnes par I'utilisateur (12). 

1 1 . Systeme selon I'une quelconque des revendications 1 a 9, dans lequel lesdits moyens d'entree sont un signal issu 
55 des moyens d'entree d'ordinateur (13). 

12. Systeme selon I'une quelconque des revendications precedentes, dans lequel lesdits moyens de traitement de 
transformation d'image (5) sont programmes afin de mettre en application les deux equations suivantes : 
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x _ R{uA-vB+mRs\n$s\nd} 

5 R{uC-vD+mRs\n$s\nd} 

<]u 2 +\/*+m 2 Fp 

ou: 

JO 

A = (cosOcosS - sinOsinScosp) 



B = (sinOcos5 - cos<I>sin6cosP) 



C = (cosOsin5 - sinOcosScosp) 



D = (sin<X>sin5 - cosOcosScosp) 

et ou : 

25 R = rayon du cercle image 

P = angle zenithal 

5 = angle azimutal dans le plan image 
O = angle de rotation du plan objet 
m = gross issement 
30 u, v = coordonnees du plan objet 

x, y = coordonnees du plan image 

13. Procede" permettant de f ournir des vues corrig6es en perspective d'une partie choisie d'une image optique capture 
a I'aide d'un objectif grand angulaire (1), I'image optique recue 6tant d6form6e p le proc6de comprenant : 

35 

la fourniture d'un signal numSrise" correspondant a ladite image optique ; 
le choix d'une partie de ladite image optique ; 

la transformation dudit signal numeris6 at in de produire un signal de sortie correspondant a une image corrig^e 
en perspective de ladite partie choisie de ladite image recue ; et 
40 I'affichage ou I'enregistrement de ladite image corrigee en perspective de ladite partie choisie ; 

CARACTERISE EN CE QUE 

ladite 6tape de transformation dudit signal numeris6 comprend le calcul de parametres de transformation 
de ladite partie choisie de ladite image, lesdits parametres de transformation calculus etant utilises afin de com- 
45 mander ladite transformation du signal num6rise* afin de produire ledit signal de sortie. 

14. Proc6d6selon larevendication 13, comprenant d'abord la reception de ladite image optique, produisantlessignaux 
correspondant a ladite image optique recue et numdrisant lesdits signaux. 

so 15. Proced6selon I'une quelconque des revendications 13 ou 14, comprenant la capture de ladite image optique avec 
un ou plusieurs objectifs ultra grand angulaires (1). 

16. Proc§d6 selon Tune quelconque des revendications 13 a 15, dans lequel ladite etape de choix de la partie de 
I'image a voir comprend le choix de I'un des 6l6ments sutvants, ou de plusieurs d'entre eux : une direction de 

55 vis6e ; I'inclinatson d'un angle de vis6e ; la rotation d'un angle de vis6e ; le panoramique dudit angle de visee ; la 

mise au point de ladite image et le grossissement de la partie choisie de I'image. 

17. Proc6d6 selon la revendication 16, dans lequel I'inclinaison dudit angle de vis6e est prevue sur au moins 1 80 °. 



15 




EP 0 971 540 B1 

18. Proced6 selon Tune quelconque des revendications 16 ou 17, dans lequel la rotation dudit angle de visee est 
prevuesur360°. 

19. ProcSde selon Tune quelconque des revendications 1 6 a 1 8, dans lequel le panoramique dudit angle de visee est 
5 prevu sur au moins 1 80 °. 

20. Precede selon ia revendication 19, dans lequel le panoramique dudit angle de visee est prevu sur 360 °. 

21. Procede selon Tune quelconque des revendications 13 a 20, dans lequel ledit choix de ladite partie de I'image a 
w voir se fait grace a des moyens de commutateur de maniputateur actionnes par I'utilisateur (12). 

22. Procede selon Tune quelconque des revendications 13 a 20, dans lequel ledit choix de ladite partie de I'image a 
voir est commande par un signal fourni par des moyens d'entree d'ordinateur (13). 

15 23. Procede selon Tune quelconque des revendications 13 a 22, dans lequel ladite transformation d'image met en 
application les deux equations suivantes : 

x _ R{uA-vB+mRsm^s\r\d} 
20 *ju 2 +v 2 +m 2 Fp 



y = R{uC-vD+mRs\r\$&\nB] 
V^+v 2 +m 2 F? 

25 

ou : 

A = (cosOcos5- sinOsinScosfi) 

30 

B = (sinOcos5 - cosO>sin8cosP) 
35 C = (cosOsin8 - sinOcos5cosP) 

D = (sinOsin8 - cosOcos6cosP) 

40 et ou : 

R = rayon du cercle image 
P = angle zenithal 

5 = angle azimutal dans le plan image 
45 o = angle de rotation du plan objet 

m = grossissement 
u, v = coordonnees du plan objet 
x, y = coordonnees du plan image 

so 24. Procede selon Tune quelconque des revendications 13 a 23, dans lequel une pluraiite de parties de ladite image 
sont choisies pour I'affichage et sont present6es simultanement ou consecutivement. 

25. Procede selon I'une quelconque des revendications 13 a 24, dans lequel I'image est vue de maniere interactive 
en repetant les etapes de choix, de transformation et d'affichage de ladite partie de I'image. 

55 

26. Procede'selon la revendication 13, dans lequel ladite etape de transformation de I'image est basee sur les carac- 
teristiques d'objectif de I'objectif grand angulaire (1). 
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27. Procede selon la revendication 26, dans lequel I'etape de transformation est basee sur I'invariabilite de I'angte 
d'azimut et sur la projection equidistante. 

28. Procede selon la revendication 13, dans lequel I'etape de transformation de I'image est realisee a des vitesses de 
5 video en temps reel. 

10 
15 
20 
25 
. 30 
35 
40 
45 
50 
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FIG. 5 
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